Global Convergence of Model Reference Adaptive Search for Gaussian Mixtures

While the Expectation-Maximization (EM) algorithm is a popular and convenient tool for mixture analysis, it only produces solutions that are locally optimal, and thus may not achieve the globally optimal solution. This paper introduces a new algorithm, based on the global optimization algorithm Model Reference Adaptive Search (MRAS), designed to produce globally-optimal solutions in the estimation of finite mixture models. We propose the MRAS mixture model algorithm for the estimation of Gaussian mixtures, which relies on the Cholesky decomposition to construct the random positive definite covariance matrices. In addition, we provide a theoretical proof of global convergence of the MRAS mixture model algorithm to the optimal solution for the maximization of the likelihood function of Gaussian mixtures. We conduct numerical experiments to evaluate the effectiveness of the proposed algorithms in comparison to classical EM.